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INTRODUCTION 


There  are  three  prerequisites  for  cost  estimating  research  and  analysis  to 
have  a  meaningful  Impact  on  the  decision  making  process  in  any  public  agency 
or  private  enterprise.  They  are: 

.  Policy 

.  Methodology  and  Tools 

.  Credible  Data 

These  prerequisites  form  a  triad  for  effective  decision  making.  Together  they 
account  for  the  necessary  and  sufficient  conditions  upon  which  good 
recommendations  and  decisions  can  be  made.  If  either  of  the  three  are  missing 
or  weak,  then  we  are  bound  to  draw  faulty  conclusions  and  make  less  than 
optimal  decisions. 

A  policy  is  needed  that  recognizes  there  is  no  free  lunch.  Weapon  systems 
like  any  other  commodity  are  being  developed,  procured  and  fielded  in  a 
resource  constrained  environment.  Official  decision  making  policy,  practices 
and  procedures  should  comprehend  this  imperative  and  establish  a  framework  for 
rationally  evaluating  resource  trade  offs  and  military  worth  or  utility. 

At  the  same  time,  rigorous,  defensible  and  reasonably  sophisticated 
methodological  approaches  and  evaluation  tools  are  necessary.  These  tools 
should  enable  analysts  and  decision  makers  to  conduct  cost  and  economic 
research  within  the  policy  framework;  lead  to  results  and  conclusions  which 
are  understandable  at  any  echelon  in  the  decision  making  establishment;  and 
allow  any  interested  party  to  repeat  the  analysis  and  come  out  with  the  same 
results. 

The  third  leg  of  the  triad  is  data.  Data  must  be  partitionable  in  the 
infinite  number  of  ways  that  questions  arise  in  the  policy  and  decision  making 
establishment.  It  also  must  be  clean  and  valid  before  it  is  used  in  the 
numerous  tools  and  models  of  our  cost  estimating  and  analysis  profession. 

The  intent  of  this  paper  is  to  investigate  the  vigor  and  health  of  this 
cost  analysis  triad  within  the  Defense  Department.  This  investigation 
reflects  an  awa  aness  of  cost  analysis  policy  development  over  the  past  two 
decades  as  well  as  an  insight  into  the  development  of  cost  estimating 
methodologies  and  tools  over  a  25  year  period.  The  strength  of  the  data  leg 
of  the  triad  is  also  investigated,  albeit,  from  a  somewhat  more  introspective 
point  of  view. 

POLICY  DEVELOPMENT 


1961  was  a  turning  point  fcr  the  thenceforth  growing  impact  of  economists 
and  cost  analysis  in  the  Defense  Department.  Cost  analysis,  cost 
effectiveness  and  program  budgeting  started  on  their  long,  uphill  journey 
towards  becoming  prime  elements  In  the  decision  making  process.  These 
concepts,  derived  primarily  from  the  thinking  of  Charles  Hitch,  were 
Implemented  by  Robert  McNamara  when  he  was  appointed  Defense  Secretary  in 
1961.  First  heralded  the  Program  Change  Control  system,  the  Planning, 
Programming  and  Budgeting  System  (PPBS)  as  we  know  it  today  was  born. 


Prior  to  this  time,  the  Defense  Department  developed  Its  force  structure 
by  starting  with  a  budget  and  then  seeking  out  possible  programs.  There  was 
no  pragmatic  way  of  relating  costs  to  weapons  systems,  tasks  and  missions. 
Time  phased  costs  of  proposed  programs  were  not  known,  studied  or  rigorously 
analyzed.  Data  needed  to  assess  properly  the  costs  or  effectiveness  of 
alternative  programs  was  not  available. 

Within  this  basic  framework,  cost  estimating  and  analysis  requirements 
began  to  be  defined  as  key  policy  and  decision  making  variables.  Some  of  the 
important  milestones  in  the  development  of  the  policy  framework  include: 

1962  -  The  Program  Change  Control  System 

Department  of  Defense  Directive  7045.1 

1964  -  Integrated  Logistic  Support 

Department  of  Defense  Directive  4100.35 

1966  -  Resource  Management  Systems  of  the  DoD 
Department  of  Defense  Directive  7000.1 

Cost  Information  Reports 

Department  of  Defense  Instruction  7041.2 

1968  -  Work  Breakdown  Structures  for  Defense  Material  Items 
Department  of  Defense  Directive  5010.20 


1969  -  Economic  Analysis  and  Program  Evaluation  for  Resource  Management 
Department  of  Defense  Instruction  7041 .3 

The  Planninq  Programming  and  Budgeting  System 
Department  of  Defense  Instruction  7045.7 


1970  - 

1971  - 

1972  - 

1973  - 


Life  Cycle  Costing  Procurement  Guide;  LCC-1 
Life  Cycle  Costing  Procurement  Guide;  LCC-2 

Major  System  Acquisition 
Department  of  Defense  Directive  5000.1 

Selection  and  Acquisition  of  ADP  Resources 
Department  of  Defense  Instruction  4105.55 

OSD  Cost  Analysis  Improvement  Group 
Department  of  Defense  Directive  5000.4 

Life  Cycle  Costing  Procurement  Guide;  LCC-3 

Design  to  Cost 

Department  of  Defense  Directive  5000.28 

Contractor  Cost  Data  Reporting 
Department  of  Defense  Directive  7000.11 
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1974  -  Level  of  Repair  Analyses 

Military  Standard  1390 

Assistant  Secretary  of  Defense  (Policy  Analysis  and  Evaluation) 
Department  of  Defense  Directive  5141.1 

1975  -  Major  System  Acquisition  Procedures 

Department  of  Defense  Instruction  5000.2 

Selected  Acquisition  Reports 
Department  of  Defense  Directive  7000.3 

1976  -  Selection  of  Contractual  Sources  for  Major  Defense  Systems 

Department  of  Defense  Directive  4105.62 

0MB  Circular  A-109 

DoD  Value  Engineering  Program 
Department  of  Defense  Directive  5010.8 

1977  -  Performance  Measurement  for  Selected  Acquisitions 

Department  of  Defense  Instruction  7000.2 

Defense  Production  Management 
Department  of  Defense  Directive  5000.34 

Uniform  Budget/Cost  Terms  and  Definitions 
Department  of  Defense  Instruction  5000.33 

1978  -  Assistant  Secretary  of  Defense  (Program  Analysis  and  Evaluation) 

Department  of  Defense  Directive  5141.1 

1979  -  Selected  Acquisition  Reports 

Department  of  Defense  Instruction  7000.3 

1980  -  DoD  In-House  versus  Contract  Commercial  and  Industrial 

Activities  Cost  Comparison  Handbook 
Department  of  Defense  Handbook  4100.33 

-  Acquistion  and  Management  of  Integrated  Logistic 
Support  for  Systems  and  Equipment 
Department  of  Defense  Directive  5000.39 

Reliability  and  Maintainability 
Department  of  Defense  Directive  5000.40 

Together,  these  policy  statements  define  a  comprehensive  framework  of 
analysis  drawing  attention  to  the  importance  of  cost  as  a  key  decision  making 
variable.  And  of  course,  each  of  the  services  have  reflected  these  policies 
within  their  own  instructions  on  a  fairly  consistent  basis. 
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Department  of  Defense  Directive  7045.1,  "The  Program  Change  Control 
System,"  (12  April  1962)  provided  Initial  policy  guidance  and  established  a 
structure  for  submitting  and  monitoring  changes  to  the  Five  Year  Defense 
Program  (FYDP).  This  directive  addressed  the  need  within  the  DoD  to  make 
program  costs  more  visible  and  thus  easier  to  control.  The  Information 
required  by  DoDD  7045.1  enabled  SECDEF  to  assess  and  evaluate  the  relative 
costs  and  merits  of  alternative  programs.  DoD  Instruction  7045.7,  "The 
Planning,  Programming  and  Budgeting  System,"  (29  October  1969)  revised  and 
updated  DoDD  7045.1  to  make  It  even  more  responsive  to  SECDEF  cost  analysis 
requirements.  The  procedural  guidance  provided  In  this  later  instruction 
(including  detailed  instructions  for  preparing  Program  Change  Requests 
(PCR's),  Program  Objective  Memorandum  (POMs),  Program  Change  Decision  (PCD) 
and  Program  Budget  Decision  (PBD)  Papers)  facilitates  "the  submission, 
analysis,  review  and  approval  of  DoD  programs  and  budgets".  DoDI  7045.7  also 
explained  in  detail  the  mechanics  of  the  PPBS  schedule  and  how  it  provides 
input  into  the  FYDP.  In  1975,  the  Secretary  of  Defense  issued  DoDI  7000.3 
"Selected  Acquisition  Reports".  This  instruction  provides  standardized 
formats  and  directions  for  reporting  comprehensive  summary  level  costs, 
quantity,  schedule  and  technical  information  on  major  systems  to  the  Secretary 
of  Defense. 

Recognition  of  the  importance  of  controlling  costs  through  increased 
visibility  and  cost/economic  analysis  was  not  limited  to  the  more  agggregate 
program  justification  levels;  the  Defense  Department  also  realized  the 
necessity  of  controlling  costs  within  the  individual  programs.  Thus,  the 
Secretary  of  Defense  and  the  Joint  Logistics  Commander  (JLC)  issued  several 
directives,  instructions  and  guides  which  provided  program  managers  within  the 
services  with  guidance  and  methodology  for  performing  cost  and  economic 
analyses  of  both  ongoing  and  proposed  programs.  These  include  DoDD  5010.20 
"Work  Breakdown  Structures  for  Defense  Material  Items"  (accompanied  by 
Military  Standard  881,  "Workbreakdown  Structures  for  Defense  Material  Item), 
DoDI  7041.3,  "Economic  Analysis  and  Program  Evaluation  for  Resource 
Management",  the  Life  Cycle  Costing  Procurement  Guides  (LCC-1,  2  &  3),  and  the 
Design  to  Cost  Guides,  as  well  as  several  directives  addressing  the 
procurement  and  acquisition  of  major  defense  systems.  Together  these 
directives,  instructions,  standards  and  guides  established  cohesive  policy 
guidance  and  the  analytical  framework  in  which  DoD  components  could 
effectively  perform  cost,  and  economic  analyses.  In  general  each  of  the 
policy  elements  provide  detailed  instructions  and  methodology  for  performing 
the  required  analyses  as  well  as  identifying  the  responsible  offices.  The 
mandatory  character  of  the  directives  ensured  more  rapid  compliance. 

In  order  to  facilitate  implementation  and  monitor  the  progress  of  defense 
cost  and  economic  analysis  policy,  the  Secretary  of  Defense  established 
several  working  organizations.  Among  these  are  the  office  of  the  Assistant 
Secretary  of  Defense  (Program  Analysis  and  Evaluation)  OASD,  (PA&E),  the  OSD 
Cost  Analysis  Improvement  Group  (CAIG)  and  the  Defense  Economic  Analysis 
Council.  The  purpose  of  these  groups  is  largely  advisory,  with  policy  making 
analysis,  methodology  development,  and  communications  responsibilities. 
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METHODOLOGY  DEVELOPMENT 


Descriptions  and  specification  of  pragmatic  methodologies  and  tools  to 
conduct  the  analysis  required  by  the  policies  noted  above  are  abundant.  The 
techniques  for  performing  cost  analysis,  cost  effectiveness  and  budget 
estimating  are  straightforward.  As  policy  requirements  have  evolved,  there 
has  been  a  complimentary  effort  In  the  literature  to  translate  state  of  the 
art  forecasting  and  estimating  techniques  to  specific  applications  within  the 
Defense  Department.  Much  of  this  applied  research  has  come  out  of  the  Federal 
Contract  Research  Centers  such  as  RAND  and  GE  TEMPO.  A  representative 
sampling  of  this  documentation  includes: 

.  Weapon  Systems  Cost  Analysis;  G.  Fisher;  July,  1956. 

.  Economics  and  Military  Operations  Research;  Charles  Hitch;  August, 
1958. 

.  Procedures  for  Development  of  a  Weapon  System  Cost  Methodology  for 
Navy  Weapons;  bonald  Clegg;  May,  1959. 

.  Use  of  the  Learning  Curve;  D.  Novick;  November,  1961. 

.  Costinq  Concepts  for  the  Defense  Proqrams  Management  System;  David 

'CurryjltarchTW: -  - - - - 

.  Project  Estimating  by  Engineering  Methods;  P.  Gallagher;  1965. 

.  An  Introduction  to  Equipment  Cost  Estimating;  J.  Large;  January,  1968. 

.  Life  Cycle  Cost  Modeling;  J.  Hamilton,  Army  Material  Command; 
December,  11966. 

.  Life  Cycle  Costinq  in  System  Acquisition;  Logistics  Management 
Institute;  November,  1969. 

.  A  Generalized  Life  Cycle  Cost  Model  for  Electronic  Equipment;  Booz, 
Allen  and  Hamilton; March,  1970. 

.  Cost  Management  Methods;  Booz,  Allen  and  Hamilton;  March,  1971. 

.  Costing  Methodology  Handbook;  Office  of  the  Comptroller  of  the  Ar»y; 
April,  1971. 

.  Guidelines  for  Cost  Estimation  by  Analogy;  C.  Trigg,  U.S.  Army 
Electronics  Command;  1973. 

.  Models  and  Methodology  for  Life  Cycle  Cost  and  Test  and  Evaluation 
Analysis;  Richard  Anderson;  July,  1973. 

.  AFLC  Operations  and  Support  Cost  Model;  AFSC,  AFLC  Wright  Patterson 
AFB;  October,  1973~ 
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.  Navy  Weapons  System  Life  Cycle  Cost  Model;  Naval  Weapons  Engineering 
Support  Activity;  September,  19>4. 

.  Logistics  Support  Cost  Model  User* r>  Handbook;  AFLC/MMOA,  Wright 
Patterson  AFB;  February,  1974. 

.  Life  Cycle  Cost  Model;  Thomas  Otto,  Army  Electronics  Command:  July, 
1975. 

.  Life  Cycle  Cost  Model;  Joint  Tactical  Communication  Office.  TRI-TAC: 
1576.  - 

.  TRI-TAC  Life  Cycle  Cost  Model;  Joint  Tactical  Communications  Office: 

m -  - 

•  NAVELEX  Life  Cycle  Cost  Model;  Naval  Electronics  Systems  Command, 

Cost  Analysis  Group;  1576. 

•  TRI-TAC  Life  Cycle  Cost  Model;  Joint  Tactical  Communications  Office; 
‘September,  1981. 

The  above  documents  provide  only  a  brief  sample  of  the  extensive  amount  of 
study  and  research  In  the  areas  of  cost  and  economic  analysis.  Nonetheless, 
they  serve  to  illustrate  the  development  and  evolution  of  cost  analysis 
methodology  over  the  past  26  years. 

During  the  late  1950's  and  into  the  1960 ' s  cost  estimating  methodology  was 
relatively  simplistic  and  limited  to  specific  applications.  Integrated  and 
comprehensive  life-cycle  cost  analysis  methodology  as  we  know  it  today,  was 
not  available  to  DoD  decision  makers. 

Charles  Hitch  in  Economics  and  Military  Operations  Research  Indicated, 
military  and  the  government  officials  frequently  approached  an  acquisition 
with  a  requirement  analysis.  In  this  scenario,  staff  officers  would  establish 
a  set  of  "required"  tasks  or  performance  characteristics  based  on  "military 
judgement"  or  "needs";  cost  considerations  were  typically  excluded  from  the 
initial  analysis.  Costs  were  later  computed  in  terms  of  budget,  or  manpower 
required  to  meet  the  performance  parameters,  e.g.,  costs  analysis  was  an 
outgrowth  of  the  requirements  analysis.  Even  at  this  early  date,  Hitch  argued 
for  the  application  of  operations  research  and  economic  analysis  to  major 
defense  acquisition  decisions. 

The  growing  importance  of  inflation  and  budgetary  considerations  during 
the  1960 's  was  reflected  in  increasing  requirements  of  DoD  policy  and 
guidance.  These  development  in  turn  encouraged  both  the  development  and 
implementation  of  more  sophisitacated  cost-estimating  methodologies.  The  U.S. 
Air  Force  was  a  pioneer  in  the  field  of  weapon  systems  cost  analysis.  In 
order  to  more  accurately  predict  system  acquisition  costs,  the  Air  Force 
(followed  shortly  by  the  other  services)  introduced  several  methodologies 
including: 
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.  learning  curve  analysis 

.  statistical  regression  analysis 

.  Industrial  engineering  standards 

.  rates  and  factors 

.  specific  analogies 

.  estimating  relationships 

A  thorough  dlscussslon  of  these  and  other  approaches  to  cost-estimating 
appears  In  the  Air  Force  Systems  Command  Manual  "Cost  Estimating  Procedures" 
dated  28  November  1967,  as  well  as  several  other  of  the  previously  referenced 
documents.  The  development  and  refinement  of  these  methodologies  represent  a 
substantial  contribution  to  the  field  of  cost  analysis.  However,  these 
seminal  techniques  concentrated  on  controlling  production  costs.  As  cost 
analysis  achieved  widespread  application,  cost  estimators  recognized  the  need 
for  a  tool  or  process  which  would  both  bring  together  the  different  estimating 
elements  and  enable  the  government  to  determine  the  total  cost  of  an  Item  or 
system  over  its  full  life. 

<v 

The  DoD  concept  of  Life  Cycle  Cost  evolved  in  the  late  1960* s  to  meet  this 
requirement.  Life  Cycle  Costing  may  be  defined  as  "the  process  of  listing  or 
determining  the  individual  costs  of  an  item  or  system  frcn  development  and 
production  costs,  through  operating  and  support  cost,  to  and  Including 
disposal  costs."  The  late  1960s  and  early  1970s  witnessed  the  development  of 
several  life-cycle  cost  models  (LCCMs).  These  models  were  based  on  work 
breakdown  structures  which  dlsaggregrate  the  system  to  subsystem  and  component 
levels  and  a  cost  breakdown  structure  which  disaggregates  cost.  These  LCCMs 
accounted  for  time  phasing  of  costs,  and  inflation  among  other  relevant 
issues.  In  general,  these  early  models  were  not  automated,  and  were  oriented 
toward  particular  equipment  groups,  e.g.,  aircraft,  electronics  gear, 
artillary.  Furthermore,  earlier  models  concentrated  on  estimating  production 
costs  while  addressing  development  and  support  costs  to  a  lesser  degree  of 
sophistication. 

Throughout  the  1970s,  DoD  LCC  policies  became  more  oriented  to  controlling 
operations  and  support  costs  as  reflected  by  the  several  directives  dealing 
with  logistics  management.  Additionally,  there  was  a  concerted  effort  to  make 
the  LCCMs  more  generic  and  useable.  Consequently,  recent  LCCMs  have  become 
increasingly  comprehensive,  addressing  all  phases  of  the  equipment  life-cycle 
with  a  significant  degree  of  sophistication.  In  general,  these  LCCMs  exhibit 
a  much  wider  degree  of  application  than  their  predecessors,  both  in  terms  of 
the  ability  to  handle  a  variety  of  equipment  types  and  useability;  models  are 
automated  and  allow  the  user  substantial  flexibility  in  the  choice  of  cost 
elements,  time  phasing,  and  estimating  relationships.  For  example,  the  most 
recent  revision  of  the  TRI-TAC  LCCM  lists  the  following  characteristics: 

.  Interactive  feature 

.  Easy  access  to  multiple  users  - 

with  user  confidentiality  and  data  access  controls 
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Cost  element  structure  - 
with  costs  element  definitions 


.  Incorporated  data  base  - 

default  values  for  assumptions,  pay  data, 
maintenance  scenarios 

.  Costing  modules  - 

Inf lation/dlscounting,  change  base  year,  phase 
spending  rates 

.  Diversity  of  output  reports  - 

LCC,  DTUf’C,  Annual ized  Cost,  POM 

.  Data  collection  workbook  and  record  book  - 

for  audit  trail;  for  recording  sources,  assumptions, 
working  notes 

Although  this  list  by  no  means  exhausts  the  capabilities  of  the  TRI-TAC  LCCM, 
it  serves  to  Illustrate  the  versatility  of  the  model. 

In  sum,  cost  and  economic  analysis  methodology  has  improved  significantly 
In  the  past  20-25  years,  in  response  to  the  changing  needs  of  the  DoD  economic 
environment.  The  ability  to  estimate  program  costs  has  evolved  from  simple 
requirements  analyse^  in  the  late  1950s  to  acquisition  cost  estimating 
methodologies  in  the  1960s,  to  comprehensive  life  cycle  cost  models  in  the 
1970s  and  1930s .  However,  these  models  are  only  as  accurate  as  the  data 
used  If  a  model  has  to  work  with  insufficient,  inaccurate  or  grossly 
exaggerated  data,  it  can  be  expected  that  the  model  will  generate  similarly 
Inaccurate  and  unacceptable  results. 

DATA  :ND  DATA  BAS£S 

Booz,  Alien  has  been  involved  in  the  conduct  of  cost  analysis  research  for 
the  Department  of  Defense  since  the  formation  of  a  subsidiary  devoted  to 
providing  consulting  services  to  the  federal  governement  in  1955.  This  part 
of  the  analysis  reflects  our  experience  with  respect  to  the  existence,  quality 
and  availability  of  relevant  cost  information  and  data  with  which  to  carry  out 
particular  research  efforts.  While  not  totally  comprehensive  In  terms  of  all 
the  studies  which  have  been  conducted  both  within  the  Defense  Department  or 
the  consulting  industry,  our  experience  does  represent  a  good  cross  section  of 
efforts  that  have  been  made. 

The  criteria  used  to  evaluate  the  data  that  we  have  used  over  this  period 
are  threefold: 

.  Quantity 

.  Quality 

.  Access 
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By  quantity,  we  are  referring  to  the  mere  existence  of  return  cost 
Information  for  existing  systems  and  equipments.  The  classical  approach  used 
In  conducting  an  economic  analysis  for  proposed  systems  and  concepts  Is  to  go 
to  various  data  repositories  and  pull  Information  together  that  will  answer 
the  question  —  How  much  did  It  cost  the  last  time  we  did  It?  —  and  then 
extrapolating  the  required  estimate  from  tHs  data  base,  using  the 
methodologies  and  techniques  described  previously.  Of  course,  partitioning  of 
this  Information  Is  a  critical  aspect  of  the  nature  of  the  answer.  As  policy 
and  methodologies  have  evolved,  data  requirements  have  become  ever  more 
granular  and  finely  divided.  A  cost  element  struture  of  2-3  elements  In  1960 
has  given  way  to  analyses  which  In  some  cases  require  more  than  1000  discrete 
elements  In  the  1980s. 

The  quality  dimension  addresses  the  Inherent  character  of  each  data 
point.  Here  we  are  concerned  about  the  accuracy,  validity  and  reliability  of 
data  being  used  In  particular  economic  analyses. 

The  third  criterion  has  to  oo  with  availability.  Given  that  Information 
of  some  utility  to  the  cost  analyst  exists,  what  kind  of  access  does  the 
research  team  have  to  data.  Timeliness  Is  also  a  consideration. 

With  this  perspective,  the  assessment  of  data  and  data  bases  Is  made  by 
evaluating  quantity,  quality  and  access  to  cost  Information  used  to  conduct 
five  studies  since  1968. 

1968  -  Armored  Reconnaissance  Scout  Vehicle 

1969  -  Multiplexers,  Radio  Combiners  and  Cable  Combiners 

1978  -  Over-the-Horizon  Detection,  Classification  and  Targeting 

1979  -  Landing  Vehicle  Tracked  -  Experimental 

1981  -  Electronic  System  Operating  and  Support  Cost  Data 

For  each  of  these  cost  studies,  an  evaluation  of  the  data  Is  presented  using 
the  criteria  discussed  above. 


1.  Armored  Reconnaissance  Scout  Vehicle  (XM  800)  Parametric  Design/Cost 
Effectiveness  Study  -  1968  U.S.  Army  Tank  Automotive  Command 
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This  was  a  fairly  extensive  effort  finally  documented  In  eight  volumes 
addressing  requirements  synthesis  for  various  operational  scenarios  and 
simulations,  development  of  specific  design  and  performance  characteristics 
and  the  conduct  of  analysis  leading  to  the  selection  and  recommendation  of  a 
candidate  conceptual  vehicle.  A  detailed  life  cycle  cost  estimate  for  the 
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candidate,  eight  alternative  configurations  and  six  reference  vehicles  already 
In  the  operating  Inventory  was  prepared.  The  search  for  data  encompassed  Anqy 
cost  records,  vehicle  system  manufactures,  and  subsystem  manufactures.  Visits 
were  made  to  six  firms  to  obtain  data,  with  meager  results  due  to  the 
competitive  environment  and  the  proprietary  nature  of  the  Information. 
Subsystem  data  was  obtained  for  current  production  Items.  Data  for  this  study 
Is  rated  In  all  three  areas:  research  and  development;  production;  and 
operation  and  support.  Five  of  the  nine  ratings  are  poor  reflecting  lack  of 
data  or  access.  The  Army  did  have  good  production  cost  data  on  the  reference 
vehicles,  but  all  other  elements  were  eventually  estimated  using  elements  of 
production  cost  as  the  dependent  variable  and  a  set  of  forty  detailed 
assumptions.  In  the  operation  and  support  area,  the  response  to  requests  for 
historical  data  was  generally  of  the  nature  —  "Wnen  you  find  out  what  that 
costs,  I  would  like  to  know!" 


2.  Acquisition  Cost  Model  for  Multiplexers,  Radio  Combiners  and  Cable 
Combiners  -  1969  U.S.  Army  Electronics  Command 
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The  objective  of  this  study  was  to  develop  cost  estimating  relationships 
and  algorithms  for  all  aspects  of  the  Investment  phase  for  various 
multiplexers,  radio  combiners  and  cable  combiners.  After  lengthy  delays, 
contractual  data  on  nine  competitive  and  sole  source  contracts  awarded  in  the 
late  1960s  representing  three  contractors  was  obtained  for  analysis.  The 
Intention  at  this  point  was  to  derive  estimators  for  all  elements  of  the 
DD633,  In  Itself  a  significant  change  in  scope  from  the  original  objective. 
The  focus  was  now  to  look  at  contractor  bids  as  opposed  to  actual  return  data 
for  all  of  the  contractor  and  government  cost  elements  associated  with  the 
investment  phase. 

Review  of  the  DD633s  revealed  significant  differences  in  the  way 
contractors  roll  up  their  bids;  and  a  number  of  judgemental  adjustments  were 
required  in  order  to  normalize  the  data.  Even  so,  the  final  result  was 
limited  to  specifying  a  model  for  estimating  unit  production  costs,  and  the 
conclusions  point  out  the  utility  of  the  model  was  limited  by  the  data  base 
used  In  the  study.  As  an  aside,  it  took  over  six  months  to  access  and  obtain 
the  contractual  documents. 

3.  Over-the-Horizon  Detection,  Classification  and  Targeting  -  1978 

Naval  Electronic  Systems  Command 
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The  objective  here  was  to  prepare  an  Independent  cost  estimate  of  the 
remaining  research  and  development,  Investment  and  support  costs  associated 
with  fielding  an  Initial  Over-the-Horlzon  Detection  Classification  and 
Targeting  capability  to  support  deployment  of  the  TOMAHAWK. 

Development  efforts  had  been  ongoing  for  sometime  and  the  required 
capability  was  resaonably  well  defined.  Investment  requirements  were 
specified  In  the  form  of  detailed  equipment  lists  and  part  numbers,  and 
vendors,  eager  to  participate  In  the  program,  readily  supplied  price  quotes 
for  various  quantities  of  off  the  shelf  components.  These  were  verifiable 
from  other  sources  Including  GSA  catalogs.  Here  again,  the  operation  and 
support  phase  was  the  least  well  defined,  particularly  In  the  maintenance  and 
software  support  areas.  In  sum,  the  quantity  and  quality  of  data  available 
had  improved  over  the  intervening  ten  years,  and  access  to  the  Information  was 
facilitated  by  the  competitive  Interest  in  providing  many  of  the  standard 
components.  This  Improved  assessment  derives  primarily  from  the  specificity 
of  performance  requirements  and  objectives. 

4.  Conceptual  Design  for  the  LVT(X)  -  1979 

Naval  Sea  Systems  Command 
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This  study  provides  a  very  direct  comparison  with  the  1968  effort  for  the 
U.S.  Army  Tank  Automotive  Command.  This  effort  also  included  a  requirements 
synthesis  for  alternative  operating  scenarios,  specification  of  design  and 
performance  characteristics  and  recommendations  concerning  a  candidate 
conceptual  vehicle. 

The  cost  estimating  and  analysis  requirements  for  this  concept  design 
effort  stand  as  a  testament  to  the  growth  of  economic  analysis  policy  in  the 
Defense  Department.  Life  Cycle  Costs  were  required  to  be  estimated  for  over 
40  discrete  elements  for  each  of  14  major  susbsystems  over  a  21  year  period. 

In  addition,  there  were  over  95  elements  requiring  separate  estimates  to  build 
up  a  design-to-unit  production  cost  estimate. 

In  evaluating  the  data  used  on  this  study  it  Is  noteworthy  that  only  two 
of  the  nine  categories  are  rated  poor.  There  has  been  significant  Improvement 
in  the  quantity  and  quality  of  information,  but  gaining  access  was  a 
significant  hurdle.  Much  of  the  data  actually  used  was  finally  obtained 
during  the  fifth  month  of  this  six  month  effort. 

5.  Electronic  System  Operation  and  Support  Cost  Data  Base  -  1981 
Naval  Electronic  Systems  Command 
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5.  Electronic  System  Operation  and  Support  Cost  Data  Base  -  1981 
Naval  [  ictronic  Systems  Command 
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As  note-  thus  far,  that  portion  of  the  life  cycle  suffering  most  relative 
to  the  evaluation  criteria  used  here  is  the  operation  and  support  phase.  This 
is  recognized  by  all  of  the  services  and  OSD,  and  a  number  of  efforts  are 
underway  to  improve  the  situation.  In  recognition  of  this  problem,  NAVELEX 
asked  us  to  investigate  the  feasibility  of  compiling  an  operation  and  support 
cost  data  base  for  a  number  of  the  electronic  systems  they  have  deployed  over 
the  last  ten  years. 

We  started  out  by  defining  a  operation  and  support  cost  element  structure 
that  included  forty-five  elements  and  identified  generic  information  sources 
from  which  to  obtain  the  required  data. 

An  assessment  was  then  made  of  the  availability,  applicability  and  quality 
of  data  which  could  be  used  later  to  develop  cost  estimating  relationships  for 
the  identified  cost  elements.  As  a  trial  run,  data  was  sought  for  more  than 
50  v  -iables  relating  to  twenty-six  of  the  cost  elements  for  two  electronic 
systems.  We  found,  through  extreme  persistence,  that  it  was  possible  to 
collect  actual  data  on  36  of  57  variables  for  one  of  the  systems  and  41  of  the 
variables  for  the  other  system.  This  was  extremely  time  consuming.  Sources 
for  many  of  the  variables  are  not  readily  accessible  data  base  systems,  but 
rather  fall  within  the  purview  of  individual  management  teams  at  multiple 
locations.  Where  automated  systems  exist,  they  tend  to  have  a  process  or 
functional  orientation  rather  than  a  system  perspective.  For  example,  the 
focus  is  on  how  many  circuit  card  modules  were  processed  or  students  trained 
in  total  as  opposed  to  being  readily  traceable  to  particular  types  of 
electronic  systems.  When  this  is  the  case,  manual  sorting  of  data  is  required 
to  bridge  the  gap. 

The  next  phase  focused  on  key  operating  and  support  cost  driver  elements 
for  a  dozen  different  systems.  Simply  determining  the  quantity  of  each  system 
deployed  on  a  annual  basis  in  an  operational  status  is  a  difficult  hurdle,  let 
alone  the  construction  of  accurate  repair  and  provisioning  data.  Access  to 
existing  data  is  a  real  issue.  It  took  over  nine  months  to  obtain  maintenance 
and  material  management  system  data  in  a  format  compatible  with  the 
specification  of  annual  operational  and  intermediate  supoport  costs.  This  was 
eventually  obtained  in  the  form  of  approximately  seven  feet  of  tabular  reports 
requiring  extensive  manual  sorting. 
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CONCLUSION 


Policy  developments  have  been  both  comprehensive  and  complimentary  to  the 
support  of  cost-effective  decision  making  in  the  Defense  Department.  DoD  has 
implemented  more  than  forty  policy  directives  since  the  advent  of  the  Program 
Change  Control  system  in  1962.  If  we  were  to  count  the  number  of  updates  and 
service  level  directives  amplifying  these  basic  policies  over  the  last  20 
years  we  must  conclude  that  a  vigorc  s  and  healthy  policy  framework  is  in 
place  for  rationally  evaluating  t ^source  trade  offs. 

The  same  can  be  said  for  the  methodologies  economists  employ  to  conduct 
cost  and  economic  analysis  for  the  Defense  Department.  The  various  models  and 
tools  available  to  us  are  sufficient  to  accomplish  the  job  required. 

Unfortunately,  I  think  we  have  a  way  to  go  in  compiling  the  data  needed  to 
accomplish  ew'  various  cost 'estimating  and  analysis  tasks.  Itrrs-clear  to  me 
that  ..data  currently  exists  to  answer  most  questions  that  arise  and  that  for 
the  most  part  it  is  partitionable  to  the  extent  required  for  particular 
applications.  The  real  issue  is  one  of  obtaining  the  data  in  a  timely  manner 
and  of  reducing  the  redundant  data  collection  effort  needed  every  time  a  cost 
effectiveness  question  arises  in  the  decision  making  arena.  .. 

\N»- 

It  is  time  to  publish  common  and  statistically  sound  data  like  those 
maintained  in  other  fields  of  economic  research.  Other  federal  departments 
regularly  publish  such  information  that  is  readily  available  to  interested 
parties.  I  submit  that  the  cost  of  preparing  and  publishing  such  data  on  a 
regular  basis  is  far  outweighed  by  the  current  practice  of  developing  unique 
data  for  each  and  every  economic  question  that  arises  within  our  community. 
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